Minimax Subsampling for Estimation and Prediction in Low-Dimensional Linear Regression

Subsampling strategies are derived to sample a small portion of design (data) points in a low-dimensional linear regression model y = Xβ+ε with near-optimal statistical rates. Our results apply to both problems of estimation of the underlying linear model β and predicting the real-valued response y of a new data point x. The derived subsampling strategies are minimax optimal under the fixed design setting, up to a small (1 + ǫ) relative factor. We also give interpretable subsampling probabilities for the random design setting and demonstrate explicit gaps in statistial rates between optimal and baseline (e.g., uniform) subsampling methods.